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Abstract 

More and more test programs are requiring high 
frequency measurements. Marshall Space Flight 
Center’s Cold Flow Test Facility has an interest in 
acquiring such data. The acquisition of this data 
requires special hardware and capabilities. This 
document provides a structured trade study approach 
for determining which additional capabilities of a 
VXI-based data acquisition system should be utilized 
to meet the test facility objectives. The paper is 
focused on the trade study approach detailing and 
demonstrating the methodology. A case is presented 
in which a trade study was initially performed to 
provide a recommendation for the data system 
capabilities. Implementation details of the 
recommended alternative are briefly provided as well 
as the system’s performance during a subsequent test 
program. The paper then addresses revisiting the 
trade study with modified alternatives and attributes 
to address issues that arose during the subsequent test 
program. Although the model does not identify a 
single best alternative for all sensitivities, the trade 
study process does provide a much better 
understanding. This better understanding makes it 
possible to confidently recommend Alternative 3 as 
the preferred alternative. 
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Background 

Human spaceflight is one of the most complex 
sociopolitical, scientific and engineering 
undertakings of all times. Sending a representative 


human being from any society to altitudes and distances 
beyond which few others have traveled says volumes 
about the society undertaking the feat as well as speaks 
to those individuals who step forward to contribute. To 
build spacecraft hardware requires a mastery of the 
sciences and a progressive engineering approach but to 
build human spaceflight hardware requires the 
development of a spacecraft worthy of representing such 
a society and capable of safely transporting the most 
precious of payloads. 

To date, the only practical propulsion method with 
sufficient energy for launching spacecraft from Earth and 
reaching orbital levels has been by combustion of 
chemical propellants. [1] This method of propulsion 
results in highly complex systems. [2] The performance 
requirements and harsh operational environments of 
these propulsion systems have pushed the design of 
components to operate near their structural and design 
limits. To mitigate these margins, experimental tests are 
used to assist in the definition of complicated three- 
dimensional flow environments as well as to determine 
hardware performance. 

During the late 1990’s, a test program was 
conducted at NASA's Marshall Space Flight Center’s 
(MSFC) Cold Flow Test Facility. The primary purpose 
of that test program was to increase the understanding of 
dynamic environments in turbines. The program 
involved developing a full-scale cold flow turbine model 
and instrumenting the rotor blades with surface-mounted 
high frequency pressure transducers. [3] The data for the 
test was monitored and collected with the Computer 
Aided Dynamic Data Monitoring and Analysis System 
(CADDMAS). [4] The CADDMAS was, at the time, one 
of the first real-time data monitoring and acquisition 
systems. This program resulted in many improvements to 
testing as well as the monitoring and acquisition of high 
frequency measurements. 

Introduction 

Data acquisition hardware has improved 
significantly over the last decade. To maintain a high 
level of capability, MSFC’s Cold Flow Test Facility 
upgraded its dynamic data monitoring and recording 
system from the previously used CADDMAS to a VXI- 
based system. The monitoring and recording capabilities 
of the system had been used to monitor and record high 
frequency measurements on two previous test programs. 
In both programs, the VXI system had proven to be a 



reliable replacement for the monitoring and 
acquisition capabilities of the previous CADDMAS 
system. Once the VXI-based system’s base 
monitoring and recording capabilities had been 
proven it was decided to perform a trade study to 
explore which, if any, additional capabilities of the 
VXI system should be utilized. 

The purpose of this paper is to present a case in 
which a trade study approach was initially performed 
to provide a requisite, or sufficiently analyzed, 
recommendation for which a decision could be made. 
[5] The paper continues by discussing the 
implementation of the chosen alternative and the 
system’s performance during a subsequent test 
program. Finally, the paper then documents 
revisiting the trade study with modified alternatives 
and attributes to address issues that arose during the 
subsequent test program. 

Trade Study Methodology 

A deviation of the SMART (Simple Multi- 
attribute Rating Technique) methodology was 
utilized for the trade study. Input was taken from 
Systems Engineering Principles and Practice [7] to 
modify the eight original steps for a SMART 
analysis, detailed in Goodwin and Wright. [6] These 
modifications provide additional steps to further 
clarify the process. The general steps are enumerated 
below: 

1. Identify decision maker(s). 

2. Define objective(s). 

3. Determine alternatives. 

4. Identify the relevant attributes. 

5. Assign metrics to the attributes. 

6. Determine a weight for each attribute. 

7. Assign values ratings for alternatives. 

8. Calculate the comparative scores. 

9. Make a provisional decision. 

10. Perform sensitivity analysis. 

1 1 . Make a requisite recommendation. 

The following sections follow these general steps 
for evaluation of the data acquisition system trade. 

Objectives and Attributes 

The VXI system had already demonstrated 
effectiveness in meeting the data monitoring and 
recording requirements for the test facility. The 
objectives for the trade study were developed through 
discussions with the test conductors and data 
analysts. The objectives were to determine which 
VXI system configuration would result in the 
simplest, most reliable, and least intrusive system that 
would continue to meet the test facility's needs. 

In order to reduce the teams time spent in a 
conference room, the systems engineer developed the 


attributes, subattributes, and subattribute measures with 
minimal team participation. The following attributes 
were selected to measure the performance of each 
alternative to meet the above stated objectives: 

Attribute 1. Complexity attributes mapped back to 
the “simplicity” objective and included the 
following subattributes: 

a. Number of subsystems 

b. Complexity of subsystems 

c. Number of interfaces 

d. Skill level required to setup 
Attribute 2. Reliability attribute mapped back to the 

“reliable” objective and included the number of 
failures per eight hours of testing. 

Attribute 3. Replacement attributes mapped back to 
the “reliable” objective and includes the following 
subattributes: 

a. Cost to replace/repair one channel 

b. Time required to replace/repair one 
channel 

Attribute 4. Setup mapped back to the “least 
intrusive” objective and includes the following 
subattributes: 

a. Cost and hours for initial setup 

b. Cost and hours for test series setup 

c. Cost and hours for daily setup. 

Attribute 5. Safety is innate in all test hardware and 

is mapped back to meeting test needs. This 
includes possible hazards to personnel and/or 
hardware. 

After the attributes were developed, a detailed 
explanation of the attributes and how they address the 
facility’s objectives was presented to the team members 
for input and discussion. To assist the discussion, it was 
necessary to have a general concept of the alternatives in 
order to discuss how these attributes could be used to 
distinguish the alternatives. 

At this point, the standard SMART methodology 
would call for using swing weight to develop the 
attribute weights. This analysis deviated from the use of 
swing weights and opted for the use of pairwise 
comparisons to weight the attributes. This choice was 
made primarily because of the number of team members 
and the simplicity of talking through only one matrix 
with all members. [7] The pairwise matrix shown in 
Exhibit 1 was formed by having a team discussion and 
comparing the relative importance of all attributes. The 
development of this matrix identified the value structure 
of the team. The pairwise matrix results were weighted 
using a rank sum method to develop the attribute 
weights. See Exhibit 2 for the rank sum results. 

While discussing the attributes with the team, it was 
determined that in-category weighting would be 
preferred rather than equally weighting each subattribute 
within their respective higher level attribute. For 
example, the Setup attribute was comprised of three 
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subattributes: Initial Setup, Test Series Setup, and 
Daily Setup. While all are important and would be 
accounted for, the Initial Setup and Test Setup are 
performed much less frequently than Daily Setup. A 
Daily Setup may be performed 100 times compared 
to one time for the Initial and Test Series setups. It 
woidd hardly make sense to weight these equally. 
This led to the decision to weight subattributes as a 
percentage of the respective attribute, as shown in 
Exhibit 5. 

Concerns were voiced as to the particular values 
used for the subattributes weights. This was 
addressed by reminding team members of the original 
objectives as well as noting that a sensitivity analysis 
woidd determine the robustness of the final 
recommendation to these subattribute weights. 


Exhibit 1. Pairwise Comparison of Attributes 
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Exhibit 2. Attributes, Ranks, and Weights. 


Attribute 

Rank 

Inverted 

Rank 

Weight 

1 ) Complexity 

a. Minimize number of subsystems 

b. Minimize complexity of subsystems 
ranked versus other alternatives 

c. Minimize number of interfaces 

d. Minimize skill level required to setup and 
use 

3 

3 

20% 

2) Reliability 

a. Minimize number of failures per eight 
hours of testing 

2 

4 

27% 

3) Replacement 

a. Minimize cost to replace/repair one 
channel 

b. Minimize time required to replace/repair 
one channel 

5 

1 

7% 

4) Safety 

a. Minimize possible hazards to people and 
hardware 

4 

2 

33% 

5) Setup 

a. Minimize cost and hours for initial setup 

b. Minimize cost and hours for test series 

c. Minimize setup cost and hours for daily 
setup 

1 

5 

13% 


Alternatives 

Three hardware alternatives were selected for 
scoring. Alternative 1 utilized no additional 
capabilities and continued using the system as a data 
monitoring and recording system only. Alternative 2 
continued using the data monitoring and recording 
capabilities as well as the signal conditioning 
capabilities. Finally, Alternative 3 continued using 
the data monitoring and recording capabilities as well 


as the signal conditioning and dedicated power supply 
capabilities. 

These alternatives are detailed below and illustrated 
in Exhibit 3. 

Alternative 1 . Utilized no additional capabilities 

a. Amplifier provides transducer excitation 
voltage 

b. Transducer converts physical phenomena to a 
measurable quantity (voltage) 

c. Amplifier provides signal conditioning - 
filtering, gain, coupling, and isolation 
between transducer and DAS (data 
acquisition system) 

d. The DAS records data and allows real time 
monitoring of signal 

e. Voltage calibration is performed by injecting 
a calibrated voltage at second amplifier input 

Alternative 2. Utilize only signal conditioning 
capabilities 

a. Amplifier provides excitation voltage to 
power transducer 

b. Transducer converts physical phenomena to a 
measurable quantity (voltage) 

c. The DAS conditions the signal, records data 
and allows real time monitoring of signal 

Alternative 3. Utilize both signal conditioning and 
power supply capabilities 

a. Dedicated power supply provides excitation 
voltage to power transducer 

b. Transducer converts physical phenomena to a 
measurable quantity (voltage) 

c. The DAS conditions the signal, records data 
and allows real time monitoring of signal 

Additional alternatives could have been generated 
by considering the many software capabilities that the 
VXI system has to offer. The team decided not to 
incorporate software alternatives because of the 
extremely large number of possible configurations and 
the fact that most foreseeable software modifications 
would not result in significant changes to the facility and 
therefore could be considered with much less rigor at a 
later time. 

Exhibit 3. Alternatives 


Alternative 1 - Utilize no additional capabilities. 
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Alternative 2 - Utilize only signal conditioning 
capabilities. 



Alternative 3 - Utilize both signal conditioning 
and power supply capabilities. 



Scoring the Alternatives 

The systems engineer initially scored each 
alternative on all attributes detailed in the Objectives 
and Attributes section. A narrative evaluation was 
provided to the team for discussion and inputs. The 
team inputs were incorporated and are detailed in 
Exhibit 4. 

All evaluations were based on quantitative 
measures with the exception of the “Complexity of 
Subsystems” and the “Skill level required to setup 
and use” subattributes. These subattributes were 
scored by the team based on the relative complexity 
of each alternative. All evaluations were then 
converted to a five-point scale with 5 being the best 
and 1 being the worst. The equivalent numeric 
evaluation of Exhibit 4 is shown in Exhibit 5. 

It should be noted that all alternatives had equal 
rank on the following subattributes: Test Series Setup 
Costs, Daily Setup Costs, and Possible Hazard to 
People. When all alternatives are scored equally 
against an attribute that attribute offers no value or 
utility to the study and can then be thrown out. [6] 
That being stated, these subattributes were carried 
through the rest of the study but did not contribute to 
the final decision. 

Calculations to Make a Selection 

The final scores were calculated in two steps. 
The first step was to calculate attribute scores. This 
was done by summing, within each attribute group, 
the products of each alternative’s score and the 
corresponding subattribute weights. Next, for each 
alternative, these attribute scores were multiplied by 
their corresponding attribute weights and summed for 
a final score. 

The attribute scores are denoted in Exhibit 5 
with a single underline. A sample attribute score 


calculation for the Complexity Attribute of Alternative 1 
is shown in Equation (1) below. 

(0.2*3)+(0.2*l)+(0.2*3)+(0.4*l) = L8 (1) 

The final scores are denoted in Exhibit 5 with a double 
underline. A sample final score calculation for the 
overall score of Alternative 1 is shown in Equation (2). 

(0.2*1. 8)+(0.27*l)+(0.07*4.5)+(0. 13*2.5)+ 

(0.33*4.8) =2M (2) 

The final scores are shown in Exhibit 5. Of the three 
alternatives, the 4.15 value is highest and therefore 
indicated that Alternative 3 - Utilize both signal 
conditioning and power supply capabilities best meets 
our values structure and should be chosen. 

Exhibit 4. Evaluation of Alternatives 



Alternatives 

Attributes 

1 

2 

3 

Complexity 




Number of Subsystems 

5 subsystems 

4 subsystems 

4 subsystems 

Complexity of Subsystems 

Jsing all amplifier 
capabilities is 
extremely complex 

.imiting amplifier use 
simplifies overall 
system. Subsystems 
have more dedicated 
purposes 

All subsystems have a 
Jedicated use 

Number of interfaces 

4 - cal to amp, amp to 
ransducer, transducer 
o amp, amp to DAS 

3 - cal to DAS, amp 
to transducer, 
ransducer to DAS 

3 - cal to DAS, power 
supply to transducer, 
ransducer to DAS 

Skill level required to setup 
and use (scale 1 to 5) 

5 - requires in-depth 
knowledge of 5 
subsystems and 4 
nterfaces 

3 - requires 
knowledge of 4 
subsystems and 3 
interfaces 

2 - requires 
knowledge of 3 
subsystems and 4 
nterfaces but the 
rawer supply is less 
nvolved than the 
amplifier 

Reliability 




Number of failures per 8 hours 
of testing 

).4 - 2 failures per 5 
lays 

0.1 - 1 failure per 10 
days 

105 - 1 failure per 20 
days 

Replacement 




Cost to replace/repair 1 
channel 

£3200 - 1 amp, 1/16 
DAS board, 1 
ransducer, 1/80 
calibration source 

$3200- 1 amp, 1/16 
DAS board, 1 
transducer, 1/80 
calibration source 

$2400 - 1/16 power 
supply, 1/16 DAS 
raard, 1 transducer, 
1/80 calibration 
source 

Time required to replace/repair 
1 channel 

1 minute - transducer 
las 1 connector and 
amps are individually 
eplaceable 

2 minutes - transducer 
has 2 connectors and 
amps are individually 
replaceable 

30 minutes - 
transducer has 2 
connectors and power 
supply requires 16 
channels being 
changed at once 

Safety 




Possible hazard to people 

Negligible (or 
alternative would not 
be considered) 

Negligible (or 
alternative would not 
ce considered) 

Negligible (or 
alternative would not 
ra considered) 

Possible hazard to hardware 

Low - Amplifiers 
provide isolation to 
DAS 

yledium - DAS has 
limited signal isolation 

Medium - DAS has 
imited signal isolation 

Setup 1 

Initial Setup 




Hours 

0 - currently installed 

120 hours - requires 1 
additional set of 
cables because 
excitation and signal 
cables will be 
separate as well as 
building a new 
calibration interface 

120 hours - requires 1 
additional set of 
cables because 
axcitation and signal 
cables will be 
separate as well as 
railding a new 
calibration interface 

Cost 

0 - currently installed 

$50,000 - $24,000 for 
two spare DAS 
boards +26,000 for 
new cabling and 
calibration interface 

£94,800 - $24,000 for 
two spare DAS boards 
•-26,000 for new 
cabling and calibration 
nterface+ 44,800 for 
sower supplies 

Test Series Setup 




Hours 

48 hours - 2 days to 
route cables, 1 day for 
amp setup and 3 days 
to perform checkouts 

64 hours - 4 days to 
route cables, 1 day for 
amp setup and 3 days 
to perform checkouts 
(twice as many cables 
o run) 

56 hours - 4 days to 
oute cables and 3 
days to perform 
checkouts (twice as 
nany cables to run) 

Cost 

Negligible 

Negligible 

Negligible 

Daily Setup 




Hours 

60 minutes - to perform 
calibrations with amps 

5 minutes - amps no 
onger influence cals 

i minutes - amps no 
onger influence cals 

Cost 

Negligible 

Negligible 

Negligible 
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Exhibit 5. Scoring of Alternatives 



Attribute 

Weights 

Alternatives 

Attributes 

Subattribute 

Weights 

1 

2 

3 

Complexity 

11 

0.20 

18 

M 

12 

Number of Subsystems 

0.2 


3 

4 

4 

Complexity of Subsystems 

0.2 


1 

3 

5 

Number of interfaces 

0.2 


3 

4 

4 

Skill level required to setup 
and use (scale 1 to 5) 

0.4 


1 

3 

4 

Reliability 

11 

0.27 

1 

3 

5 

Number of failures per 8 
hours of testing 

1 


1 

3 

5 

Replacement 

11 

0.07 

15 

15 

Z5 

Cost to repiace/repair 1 
channel 

0.25 


3 

3 

4 

Time required to 
replace/repair 1 channel 

0.75 


5 

5 

2 

Safety 

11 

0.33 

18 

3J) 

M 

Possible hazard to people 

0.1 


3 

3 

3 

Possible hazard to 
hardware 

0.9 


5 

4 

4 

Setup 

11 

0.13 

2J5 

3J) 

M 

Initial Setup 






Hours 

0.1 


5 

3 

3 

Cost 

0.1 


5 

3 

1 

Test Series Setup 






Hours 

0.1 


4 

2 

3 

Cost 

0.1 


3 

3 

3 

Daily Setup 






Hours 

0.5 


1 

5 

5 

Cost 

0.1 


3 

3 

3 


2.86 

3.60 

4.15 


Sensitivity 

To determine our confidence in the value of our 
selection, a sensitivity analysis was performed. Three 
approaches were considered for the sensitivity 
analysis. The first approach considered reevaluating 
the rank and weights of the five chosen attributes. 
The second approach considered reevaluating the 
subattribute weights. The third approach considered 
reevaluating each alternative’s score. 

The team decided there was no interest in 
reevaluating the scores for each alternative because 
the initial scores were primarily quantitative and were 
considered to be accurate and representative. 
However, the team decided to investigate the effects 
of modifying the five attribute weights and the 
subattribute weights. Modifying the attribute and 
subattribute weights would assist in determining if 
Alternative 3 would still be the preferred alternative 
if the value structure changed. 

Sensitivity Approach 1 - Modifying Attribute 
Weights 

When considering modifying the attribute 
weights, it was noted that Safety was initially 
weighted the highest of all attributes and that 
Reliability was initially weighted second of the 


attributes. These initial weightings were based on the test 
needs objective which includes the value that no 
unscheduled downtime would be acceptable, either due 
to failure or damage. What if the team were willing to 
trade both Hardware Safety and Reliability for both 
Replacement and Complexity with the preference given 
to Replacement? Exhibit 6 shows the modified pairwise 
comparison matrix. 

This change in weighting essentially means that the 
team would be less stringent on damage and failure if it 
were possible to make repairs or replacements quickly. 
This only slightly modified our value from “no 
downtime” to “limited downtime” and resulted in 
slightly favoring Alternative 2. Exhibit 7 shows the 
modified final scores. These results add confidence to the 
Alternative 2 recommendation by not strongly favoring 
another alternative. 


Exhibit 6. Pairwise Comparison of Attributes with 
Modified Attribute Weights 
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Exhibit 7. Sensitivity Approach 1 Result. 



Alternatives 

Attribute 

Rank 

Inverted 

Rank 

Weight 

1 

2 

3 

Complexity 

2 

4 

0.267 

0.48 

0.91 

1.12 

Reliability 

4 

2 

0.133 

0.13 

0.40 

0.67 

Replacement 

2 

4 

0.267 

1.20 

1.20 

0.67 

Safety 

3 

3 

0.200 

0.96 

0.78 

0.78 

Setup 

4 

2 

0.133 

0.33 

0.52 

0.51 


Total 

3.11 

3.81 

3.74 


Sensitivity Approach 2 - Modifying Subattribute 
Weights 

When considering modifying the subattribute 
weights, it was noted that little weight was given to the 
Cost subattributes. This was the case because the relative 
costs of cold flow test elements are small when 
compared to other test types. However, if funding were 
reduced these values would surely change. Therefore the 
change in the Cost subattributes' weights was the focus 
of the subattribute sensitivity analysis. 

This approach to sensitivity analysis only effected 
the Replacement and Setup attributes because they were 
the only attributes with the subattribute of Cost. To 
perform the analysis the previously assigned values for 
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Costs were tripled and the other subattributes were 
adjusted to ensure the within attribute percentage 
remained 100%. The analysis resulted in favoring 
Alternative 3. Exhibit 8 shows the final scores for the 
modified subattribute sensitivity analysis and again 
confidence was added to the Alternative 3 
recommendation. 

Exhibit 8. Sensitivity Approach 2 Results. 



Alternatives 

Attribute 

Weight 

1 

2 

3 

Complexity 

0.200 

0.36 

0.68 

0.84 

Reliability 

0.267 

0.27 

0.80 

1.33 

Replacement 

0.067 

0.23 

0.23 

0.23 

Safety 

0.333 

1.60 

1.30 

1.30 

Setup 

0.133 

0.48 

0.41 

0.33 


Total 

2.94 

3.42 

4.04 


Initial Recommendation 

Upon completion of the trade and sensitivity 
analysis, it was recommended that the MSFC Cold 
Flow Test Facility move forward with Alternative 3 
which was to begin utilizing both signal conditioning 
and dedicated power supply capabilities of the VXI- 
based system. A meeting was held to discuss the 
recommendation and a presentation was made to 
describe the necessary changes. During the meeting, 
a go forward plan was developed for implementing 
the necessary facility modifications. The spreadsheet 
used to perform the trade study was beneficial in 
developing the plan. 

Implementation of Alternative 3 

The necessary changes were made to the facility 
and the system over several months. Power supplies 
were installed. Cables were made and installed. Upon 
completion of modifications, the system was verified 
using the facility standards which had been used in 
the past. All checkouts indicated that the system 
would perform as expected. 

After checkouts and verifications were 
complete, a test was requested. This test included 
new instrumentation which had never previously 
been used on the VXI-based system. Unforeseen 
issues began to arise. The primary concerns were: 

1. A low common mode rejection ratio 
contributed to a decreased signal to noise 
ratio. 

2. The identification of a facility resonance 
which required some signals to be notch 
filtered prior to reaching the data acquisition 
system. 

3. The difficulty of injecting a common signal 
across all channels simultaneously made the 
system difficult to troubleshoot. 


These issues required the 3 ld alternative to be modified 
resulting in a 4 th alternative. 

Introducing a 4th Alternative 

In order to address the problems that arose during 
the testing, the amplifiers were brought back into the 
system for the channels which were experiencing the 
issues. The amplifiers provided a solution to these issues 
by improving the common mode rejection ratio, having 
the capability to notch filter around the facility resonance 
enabling data content above and below the resonant 
frequency, and to allow for simultaneous signal injection 
on channels with the amplifiers in place. 

While the amplifiers did address the issues, 
bringing them back into the system greatly diminished 
the appeal of Alternative 3. Requiring the use of 
amplifiers resulted in the following alternative 
description which was called Alternative 4. Exhibit 9 is a 
diagram of Alternative 4. 

Alternative 4. - Utilize both signal conditioning and 
power supply capabilities as well as amplifiers 

a. Dedicated power supply and amplifiers 
provide excitation voltages to power 
transducer 

b. Transducer converts physical phenomena to a 
measurable quantity (voltage) 

c. Amplifiers can provide signal conditioning 

i. Filtering 

ii. Gain 

iii. Coupling 

iv. Provide isolation 

d. The DAS records data and allows real time 
monitoring of signal 

e. Voltage calibration can be performed by 
injecting a calibrated voltage at second 
amplifier input 

Exhibit 9. Utilize both signal conditioning and power 
supply capabilities as well as amplifiers 



Alternative 4 was essentially Alternative 1 with the 
addition of external power supplies and new cabling. The 
implementation of Alternative 3 was no small task. 
When it became necessary to add amplifiers back into 
the system, two questions quickly arose: Why didn’t the 
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trade study recommend Alternative 1 in the first 
place and how would Alternative 4 perform in the 
study. 

Scoring Alternative 4 

Alternative 4 was added to the trade and the 
scores were calculated. Exhibit 10 illustrates how 
poorly Alternative 4 scored. This was no surprise. As 
was previously stated, Alternative 4 was essentially 
Alternative 1 with the addition of external power 
supplies and cabling. During the initial trade, 
Alternative 1 was rated the lowest and now with the 
additional complexity and costs as Alternative 4 it 
scores even lower. 


Exhibit 1 ((.Scoring of Alternatives with 
Alternative 4 



Attribute Weights 

Alternatives 

Attributes 

Subattribute 

Weights 

1 

2 

3 

4 

Complexity 

11 

0.20 

! 8 

M 

A2 

14 

Number of Subsystems 

0.2 


3 

4 

4 

3 

Complexity of Subsystems 

0.2 


1 

3 

5 

1 

Number of interfaces 

0.2 


3 

4 

4 

1 

Skill level required to setup 
and use (scale 1 to 5) 

0.4 


1 

3 

4 

1 

Reliability 

11 

0.27 

1 

3 

5 

3 

Number of failures per 8 
hours of testing 

1 


1 

3 

5 

3 

Replacement 

11 

0.07 

4J> 

4J5 

Z5 

2.25 

Cost to replace/repair 1 
channel 

0.25 


3 

3 

4 

3 

Time required to 
replace/repair 1 channel 

0.75 


5 

5 

2 

2 

Safety 

11 

0.33 

4T5 

32 

32 

32 

Possible hazard to people 

0.1 


3 

3 

3 

3 

Possible hazard to hardware 

0.9 


5 

4 

4 

4 

Setup 

11 

0.13 

2J5 

M 

32 

17 

Initial Setup 







Hours 

0.1 


5 

3 

3 

3 

Cost 

0.1 


5 

3 

1 

1 

Test Series Setup 







Hours 

0.1 


4 

2 

3 

2 

Cost 

0.1 


3 

3 

3 

3 

Daily Setup 







Hours 

0.5 


1 

5 

5 

1 

Cost 

0.1 


3 

3 

3 

3 


2.86 

3.60 

4.15 

2.76 


The 6th Attribute - Capability 

As suggested by Goodwin and Wright, any 
conflict between the analytic rankings and holistic 
rankings should be evaluated. This suggests that an 
important element of the problem has not been 
captured. [6] Indeed that was the case. When 
considering the initial objectives the VXI-based 
system was assumed to have sufficient monitoring 
and acquisition capabilities and the trade was 
performed to determine which VXI system 
configuration would result in the simplest, most 
reliable, and least intrusive system that would 
continue to meet the test needs. 


The new instrumentation changed all of this. The 
previous history of success with the VXI-based system 
was not applicable anymore. Without the additional 
capabilities of the amplifiers the VXI-based system could 
no longer meet the test needs. In order to capture this an 
additional attribute was necessary. That attribute was 
called Capability and referred to the systems capability 
of adapting to accommodate future needs. 

The trade analysis was repeated to include all four 
alternatives with the incorporation of the Capability 
Attribute. Exhibit 1 1 illustrates the alternative scores as 
well as the final scores. Alternative 3 was still the 
slightly preferred solution over Alternatives 4 and 1 . This 
seems impossible because Alternative 3 alone could not 
meet the objectives. This is the result of the additional 
trades offs in which Alternative 3 is still the preferred 
alternative. 

In order to develop a better understanding, the same 
attribute sensitivity study which was performed on the 
initial trade was again applied for the revised model. In 
this application, the Capability Attribute was maintained 
as the primary attribute followed by the Replacement and 
Complexity attributes. The results shown in Exhibit 12 
indicate that unlike the previous model, this model does 
not indicate Alternative 3 as a clear cut decision. 
Depending on the value structure chosen Alternatives 1, 
3 or 4 may possibly be chosen. 

Conclusion and Recommendations 

The trade study process outlined in Goodwin and 
Wright [6] was followed to provide an informed 
recommendation as to which system capabilities should 
be utilized. The process was structured and logical and 
can be followed by those wishing to reproduce it. The 
original recommendation was based on insufficient 
attributes developed from inadequate objectives and 
resulted in a less than desired outcome. The model was 
modified to address the previous models shortcomings 
and the new model was still unable to identify a single 
best alternative. Although the model was unable to 
identify a single best alternative, the process did provide 
a much better understanding. That better understanding 
made it possible to make the recommendation to set the 
test facility up with the intent to use Alternative 3. 
Should the need for additional capability be necessary it 
would then be possible to include only those additional 
elements which are necessary to meet the test objectives. 
This approach will utilize the benefits of both 
Alternatives 3 and 4 while minimizing the drawbacks to 
Alternative 4. 

It should also be noted that without the earlier 
recommendation for and implementation of Alternative 3 
it would not be possible to utilize the benefits because 
the additional power supplies and cables would not be 
available. 
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Exhibit 11. Final Scores with Alternative 4 and 
Capability Attribute 



Attribute 

Weights 

Alternatives 

Attributes 

Subattribute 

Weights 

1 

2 

3 

4 

Capability 

11 

0.24 

4 

1 

1 

5 

Addresses changing 
requirements 

1 


4 

1 

1 

5 

Complexity 

11 

0.14 

18 

3A 

12 

14 

Number of 
Subsystems 

0.2 


3 

4 

4 

3 

Complexity of 
Subsystems 

0.2 


1 

3 

5 

1 

Number of interfaces 

0.2 


3 

4 

4 

1 

Skill level required to 
setup and use (scale 
1 to 5) 

0.4 


1 

3 

4 

1 

Reliability 

11 

0.19 

1 

3 

5 

3 

Number of failures 
per 8 hours of testing 

1 


1 

3 

5 

3 

Replacement 

11 

0.05 

3J5 

15 

3J5 

2.75 

Cost to replace/repair 
1 channel 

0.7 

5 


3 

3 

4 

3 

Time required to 
replace/repair 1 
channel 

0.2 

5 


5 

5 

2 

2 

Safety 

11 

0.29 

18 

3J) 

3,9 

M 

Possible hazard to 
people 

0.1 


3 

3 

3 

3 

Possible hazard to 
hardware 

0.9 

0 


5 

4 

4 

4 

Setup 

11 

0.10 

Z5 

19 

18 

1Z 

Initial Setup 







Hours 

0.1 


5 

3 

3 

3 

Cost 

0.1 


5 

3 

1 

1 

Test Series Setup 







Hours 

0.1 


4 

2 

3 

2 

Cost 

0.1 


3 

3 

3 

3 

Daily Setup 







Hours 

0.5 


1 

5 

5 

1 

Cost 

0.1 


3 

3 

3 

3 


3.18 

2*95 

3*43 

3*37 


Exhibit 12. Sensitivity Analysis to Attribute Ranks 

Alternatives 


Attribute 

Rank 

Inverted 

Rank 

Weight 

1 

2 

3 

4 

Capability 

1 

6 

0.29 

1.14 

0.29 

0.29 

143 

Complexity 

3 

4 

0.19 

0.34 

0.65 

0.80 

0.27 

Reliability 

5 

2 

0.10 

0.10 

0.29 

048 

0.29 

Replacement 

2 

5 

0.24 

0.83 

0.83 

0.83 

0.65 

Safety 

4 

3 

0.14 

0.12 

0.19 

0.18 

0.08 

Setup 

6 

1 

0.05 

0.69 

0.56 

0.56 

0.56 


Total 

3.22 

2.80 

3.13 

3.27 
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